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The  Education  Subcommittee  of  the 

Joint  Legislative  Commission  on 

Government  Operations 
North  Carolina  General  Assembly 
State  Legislative  Building 
Raleigh,  NC   27611 

Gentlemen: 

It  is  a  pleasure  to  submit  this  report  of  my 
evaluation  of  the  CDP  appeals  process.   I  hope  that  I  have 
responded  to  your  charge  satisfactorily  and  that  my 
findings  and  recommendations  will  prove  useful  to  you  in 
the  months  ahead. 

You  can  expect  some  redundancy  as  I  have  restated  sone 
ideas  to  make  each  of  the  major  sections  stand  on  its  own. 
Hopefully  this  will  facilitate  your  reading,  even  though 
it  lengthens  the  report. 

I  appreciate  the  courtesies  extended  to  me  in  the 
conduct  of  this  study  by  board  members,  attorneys,  and 
educators  in  several  pilot  districts  which  I  visited  and 
by  members  of  the  Department  of  Public  Instruction.   They 
could  not  have  been  more  cooperative  and  helpful.   The 
arrangements  for  my  visits  which  Joan  Rose,  contract 
administrator,  worked  out  were  also  most  helpful  and 
appreciated. 


Sincerely  yours, 
Richard  M.  Brandt 
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EXECUTIVE  SUMMARY 

The  primary  conclusion  of  this  study  of  the  pilot 
program  appeals  process  is  that  it  works  well  at  the 
initial  peer  panel  level,  but  it  has  serious  shortcomings 
at  the  board  hearing  stage.   The  appeals  procedure  adopted 
by  the  State  Board  of  Education  in  May,  1987,  limits  the 
panel  review  to  (a)  whether  proper  procedures  are  followed 
in  the  evaluation  of  an  educator  and  (b)   whether  the 
recommendation  is  arbitrary  or  capricious. 

The  statute  establishing  the  pilot  program,  GS115C- 
3  63,  prescribes  the  composition  of  a  three-member  panel  of 
trained  evaluators  so  that  both  parties  have  equal 
influence  in  the  selection  process.   Hearings  are 
structured  so  both  parties  have  equal  opportunity  to 
present  their  cases.   Panelists  review  the  performance 
appraisal  information  used  to  make  the  summative  ratings 
and  determine  if  it  is  reasonably  consistent  with  those 
ratings  and  gathered  according  to  the  prescribed 
procedures.   This  review  provides  a  second  opinion  by  a 
panel  of  professionals  of  the  same  judgment-call  based  on 
the  same  data  as  was  used  in  the  original  evaluation. 
Supposedly  the  panel  as  a  whole  is  neutral  because  both 
parties  were  equally  involved  in  the  choice  of  panel 
members . 

There  is  some  numerical  imbalance  due  to  multiple 
evaluators.   This  imbalance  does  not  impair  the 
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fundamental  fairness  because  both  sides  are  guaranteed 
equal  time  for  presentation.  It  should  be  recognized, 
however,  that  it  can  create  a  feeling  of  isolation  and 
intimidation  for  evaluatees. 

Whereas  the  process  has  worked  quite  well  at  the 
initial  peer  panel  stage  and  the  number  of  cases  being 
appealed  to  school  boards  has  not  been  great  so  far,  those 
that  have  reached  the  boards  have  not  gone  well. 
Potential  problems  lie  ahead  that,  if  not  resolved, 
threaten  to  undermine  the  performance  appraisal  system  and 
career  development  program. 

The  root  of  the  difficulty  is  the  lack  of  clear 
guidelines  for  school  board  action  and  training  of  those 
who  participate  in  the  hearings.   The  primary 
specification  for  these  hearings  so  far  is  that  each  party 
is  entitled  to  counsel.   No  limits  are  set  on  what 
evidence  can  be  considered,  no  specifications  of  the 
criteria  on  which  judgments  are  to  be  made,  and  little 
direction  provided  on  the  scope  of  the  hearing.   Although 
the  absolute  numbers  are  not  great,  almost  half  of  the 
panel  review  decisions  have  been  overturned.   Reasons  for 
some  of  these  decisions  have  little  to  do  with  the  data 
provided  by  the  carefully  developed  appraisal  system.   Too 
often,  other  considerations,  such  as  the  perceived 
political  influence  of  the  parties  involved,  were  the 
primary  factors. 
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The  major  recommendations  of  this  study,  therefore, 
are  that  (a)  board  members  who  are  involved  in  the  appeals 
hearing  process  be  well  trained  in  the  PAS  and  the  appeals 
process  and  (b)  the  board  review  be  focused  on  the  same 
data  and  guidelines  as  the  earlier  peer  panel  review. 

To  allow  the  board  complete  discretion  to  establish 
its  own  guidelines  and  procedures  is  to  undermine  an 
important,  carefully  developed  system  of  performance 
appraisal.   Boards  of  education  represent  the  public 
interest  and,  as  such,  they  ordinarily  should  have  great 
latitude.   In  this  case,  however,  they  should  either  let 
the  professionals  make  the  judgment;  or,  better  yet,  take 
the  time  and  effort  to  be  trained  in  the  appeals  process, 
including  the  TPAI,  and  double  check  the  quality  of  the 
summative  judgments. 

The  system  is  in  place.   The  number  of  complaints  is 
modest  at  this  point.   Teachers  need  a  fair  grievance 
procedure  against  potential  arbitrariness.   This  system  is 
structurally  sound,  I  believe,  and  with  the  recommended 
improvements  should  provide  that  safeguard. 
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INTRODUCTION 

This  report  describes  a  review  of  the  appeals  process 
for  participants  in  the  North  Carolina  career  development 
pilot  program,  along  with  findings  and  recommendations. 

The  charge  by  the  Education  Subcommittee  of  the  Joint 

Legislative  Commission  on  Government  Operations  was  to: 

o    evaluate  the  process  by  which  a  person  aggrieved 
by  the  teacher  or  principal  performance 
appraisals  may  appeal 

o    determine  the  fairness  of  the  process  for  all 
parties  involved 

o    determine  the  extent  to  which  the  local  boards, 
in  reviewing  the  performance  appraisals,  base 
their  decisions  on  information  obtained  through 
the  Teacher  Performance  Appraisal  Instrument 
(TPAI)  and  the  Principal  Performance  Appraisal 
Instrument  (PPAI) 

o    determine  whether  board  members  have  sufficient 
training  to  review  fairly  and  completely  the 
performance  appraisals 

The  study  was  conducted  in  June,  1988,  by  visiting  and 

interviewing  people  in  several  pilot  districts: 

Alexander,  Buncombe,  Burlington,  Haywood,  and  New  Hanover. 

In  all  58  persons  were  interviewed,  mostly  in  person,  a 

few  by  phone.   They  were  distributed  as  follows  according 

to  their  roles  in  the  profession  and/or  the  appeals 

process: 

3  school  superintendents 

4  associate  or  assistant  superintendents 

6  school  board  members  (including  the  executive 
director  of  NCSBA) 


6  lawyers  (including  school  attorneys,  NCAE  attorneys 

and  an  assistant  attorney  general) 

2  coordinators  of  career  development  pilot  programs 

6  observer  evaluators 

8  teacher  appellants 

5  principals 

15  panel  members 

3  others  (DPI  personnel  director,  school  law  experts) 

Approximately  half  of  the  interviews  were  conducted  in 

small  groups,  the  remainder  with  individuals.   The 

interviews  with  eight  teachers  who  appealed  were  all 

conducted  individually.   Upon  my  request,  they  were 

selected  so  as  to  provide  a  balance  between  those  who  were 

successful  and  those  who  were  unsuccessful  with  their 

appeals.   They  were  also  divided  between  those  who 

experienced  only  the  panel  review  and  those  who  also  had  a 

school  board  hearing.   Principals  and  panel  members 

interviewed  were  not  necessarily  those  involved  with  the 

particular  teacher  appellants  interviewed.   I  was 

interested  in  soliciting  a  range  of  opinion  from  various 

segments  of  the  educational  community,  rather  than  in 

investigating  particular  cases. 

In  addition  to  the  interviews,  I  studied  a  number  of 

documents  and  relevant  school  laws  including  the 

following: 

o    DPI  appeals  training  materials:   video-tape  for 

training  panelists.  Appeals  Procedures  adopted  by 


the  NC  Board  of  Education,  and  other  relevant 
materials  distributed  in  May,  1988 

o    Local  district  appeals  process  descriptive 
materials 

o    Sample  reports  of  appeals  panels 

o    Relevant  statutes  and  case  notes: 

115C-45   Judicial  functions  of  the  board 

115C-302   Salary  and  vacation 

115C-305   Appeals  to  board  of  education  and  to 

superior  court 
115C-325   System  of  employment  for  public  school 

teachers 
115C-326   Performance  standards  and  criteria  for 

professional  employees 
115C-3  63   Career  Development  Pilot  Program 

Needless  to  say,  a  sound  evaluation  of  the  appeals 

process  cannot  occur  without  a  thorough  understanding  of 

the  career  development  pilot  program  and  the  performance 

appraisal  system  on  which  it  is  based.   The  judgments 

presented  in  this  study  of  the  appeals  process,  therefore, 

are  derived  in  part  from  the  understandings  and  sentiments 

I  and  my  colleagues  expressed  in  our  recent  report  on  the 

performance  appraisal  instruments.* 


♦Richard  M.  Brandt  (Chair) ,  Daniel  L.  Duke,  Russell  L. 
French,  and  Edward  F.  Iwanicki,  A  Review  with 
Recommendations  of  the  North  Carolina  Teacher  Performance 
Appraisal  Instrument.   Report  submitted  to  the  Education 
Committee  of  the  Joint  Commission  on  Governmental 
Operations,  North  Carolina  General  Assembly,  May  21,  1988, 


THE  NATURE  OF  APPEALS 
The  career  ladder  establishes  levels  of 
differentiation  that  reflect  "individual  teachers*  and 
administrators'  initiative  and  desire  to  strengthen  their 
professional  abilities  and  their  success  in  doing  so."* 
The  plan  permits  educators  to  be  promoted  to  advanced 
status  and  pay  if  they  meet  specified  performance 
criteria.   It  also  provides  for  reclassification  of 
individuals  from  higher  to  lower  status  for  failure  to 
maintain  acceptable  performance  standards. 

The  appeals  process  was  prescribed  by  the  General 
Assembly  for  candidates  who  were  not  recommended  for 
higher  status  and  for  those  persons  who  were  recommended 
for  reclassification  to  a  lower  status.   It  allows  such 
individuals  to  request  a  review  of  the  recommendation  by  a 
panel  of  three  trained  evaluators  and,  if  not  satisfied 
with  the  review,  to  request  a  hearing  before  the  local 
school  board. 

Pilot  districts  developed  their  own  procedures  for  use 
during  the  1985-86  school  year.   In  order  to  achieve 
relative  uniformity  across  the  districts,  however,  a 
subcommittee  of  the  State  Steering  Committee  was  assigned 
the  task  in  fall,  1986,  of  designing  a  procedure  that 


♦North  Carolina  Career  Development  Program  Pilot:   Appeals 
Panel  Training.   Raleigh:   Department  of  Public 
Instruction,  May,  1988,  p.  5). 


could  be  implemented  statewide.   The  State  Board  of 
Education  adopted  an  appeals  procedure  in  May,  1987,  which 
became  effective  immediately. 

This  appeals  procedure  limits  the  review  conducted  by 
the  appeals  panel  to  determining  (a)  whether  proper 
procedures  were  followed  in  the  evaluation  of  a  teacher 
and  (b)  whether  the  recommendation  to  deny  career  status 
was  arbitrary  or  capricious. 

FINDINGS 
Number  of  Appeals  and  Results 

According  to  a  Department  of  Public  Instruction 
memorandum  dated  January  11,  1988,  a  total  of  65  appeals 
were  made  to  the  panels  in  1987,*  approximately  1  percent 
of  all  evaluation  cases.   Of  those  who  in  years  of 
teaching  in  their  school  district  were  eligible  for  career 
status  II  (CSII),  the  65  appeals  represent  an  estimated  7 
percent  of  those  who  did  not  achieve  it.   More  than  two- 
thirds  of  the  appeals  were  from  three  of  the  larger 
districts:   Buncombe  12,  Burke  12  and  New  Hanover  22. 
Haywood  County  with  7  appeals  was  the  only  other  district 
with  more  than  3  cases.   Five  districts  had  no  appeals. 

Approximately  four  times  as  many  appeals  resulted  in 
the  original  decision  being  upheld  by  the  panel  as  those 


♦Information  regarding  appeals  cases  in  Charlotte 
Mecklenburg  were  not  available  and  not  included  in  these 
numbers. 


that  were  overturned  (51  to  13) .   Almost  half  of  those 
upheld  were  then  appealed  to  the  board.   Of  the  24 
appealed  to  the  board,  12  were  overturned. 

Attachments  to  the  January  11  memorandum  reported 
appeals  procedures  training  activities  in  the  pilot  units, 
a  survey  of  problems  and  recommendations  cited  from  each 
of  them,  and  a  recommendation  from  the  Appeals 
Subcommittee  (November  30,  1987)  to  the  State  Steering 
Committee  that  the  procedures  not  be  changed  but  that 
additional  training  and  training  materials  were  needed  for 
board  members  and  others  involved  in  conducting  appeals. 

I  generally  concur  with  the  conclusion  of  the 
Subcommittee  that  the  small  number  of  appeals  overall,  the 
low  rate  of  overturned  decisions  at  the  panel  stage,  and 
the  lack  of  widely  endorsed  suggestions  for  change  are 
indicative  of  a  sound  process.   This  judgment  seems 
particularly  reasonable  considering  the  fact  that  the 
appeals  procedures  were  being  tested  for  the  first  time, 
when  faults  might  be  most  expected  and  training  activities 
were  still  minimal. 

Nevertheless,  problems  with  the  process  were  apparent 
in  several  districts,  and  the  number  of  board  reversals  in 
particular  was  disturbing  given  the  potentially  negative 
effect  of  such  decisions  on  the  operation  of  the 
performance  appraisal  system  and  the  career  ladder.   A 
detailed  review  of  the  whole  process  seemed  needed  to 


determine  what  improvements  could  be  made  to  ensure 
fairness  to  all  parties  and  a  smoothly  functioning  appeals 
process  that  would  not  damage  the  effectiveness  of  the 
performance  appraisal  system. 
Grounds  for  Appeals 

State  guidelines  indicate  that  the  grounds  for  appeals 
can  be  (a)  procedural  violations  during  the  performance 
appraisal  process,  (b)  arbitrary  or  capricious  summative 
ratings  on  the  TPAI ,  or  both.   The  person  making  the 
appeal  states  the  grounds  for  it.   Of  the  65  appeals 
brought  to  the  panel  stage  in  1987,  only  8  were  for 
procedural  violations  alone.   A  total  of  40  were  solely 
for  arbitrary  or  capricious  ratings,  and  16  more  were 
appealed  on  both  grounds. 

In  conducting  reviews  for  possible  procedural 
violations,  panels  examined  the  records  to  ensure  that  (a) 
the  minimum  number  of  four  observations  were  made,  two  by 
the  principal  or  his  designee  and  two  by  the  observer 
evaluator;  (b)  at  least  one  observation  was  announced  and 
one  unannounced;  (c)  formal  observations  were  followed  by 
post-conferences;  (d)  formal  observations  lasted  an  entire 
instructional  period;  (e)  formative  observation  notes 
(FODIs)  were  recorded  by  observers  and  used  to  complete 
formative  observation  data  analyses  (FODAs) ;  and  (f) 
summative  performance  appraisal  conferences  were 
conducted . 


Complaints  of  arbitrary  or  capricious  ratings  were 
reviewed  by  examining  the  evidence  to  see  if  it  was 
directly  rated  to  the  employee's  performance  and/or 
summative  appraisal.   FODAs,  FODIs,  and  the  completed 
performance  appraisal  instrument  provided  key  items  of 
evidence,  along  with  such  others  as  informal  observation 
data  if  in  written  form,  memoranda  or  letters  that  were 
shared  at  or  prior  to  the  summative  evaluation,  school 
handbooks  defining  rules  and  regulations,  evaluatee's 
written  rebuttal,  the  letter  of  appeals,  and  whatever 
either  party  said  that  related  directly  to  the  employee's 
performance  and/or  summative  appraisal.   Only  data 
collected  during  the  current  year  and  shared  with  the 
employee  prior  to  and/or  at  the  time  of  the  summative 
evaluation  were  allowable.   Panels  typically  reviewed  data 
related  directly  to  the  specific  functions  in  question  to 
judge  whether  or  not  they  were  consistent  with  the  ratings 
given.   Their  task  was  to  determine  whether  a  reasonable 
decision  was  made  based  on  the  evidence  available  or 
whether,  instead,  the  ratings  that  led  to  a  denial  of 
career  status  were  arbitrary  or  capricious.   The  panel's 
task  was  to  examine  the  evaluation  record  and  other 
relevant  evidence  and  determine  what  the  record  showed. 
Panel  members  were  to  receive  copies  of  FODAs  and  the  TPAI 
a  few  days  before  the  hearing  in  order  to  study  them  ahead 
of  time.   The  hearing  included  the  panel,  the  appellant, 
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and  the  person  or  persons  whose  decision  was  contested. 
All  persons  involved  in  the  evaluation  had  a  right  to  be 
at  the  hearing.   Each  party  could  have  one  person  from  the 
educational  community  accompany  him,  but  this  person  was 
not  permitted  to  speak.   Local  districts  were  encouraged 
to  develop  their  own  procedures  and  schedules  for  the 
hearing  but  within  the  rather  specific  guidelines 
established  by  the  State  Board  of  Education.   In  all 
fundamental  ways,  I  found  remarkable  consistency  from  one 
district  to  another  in  the  procedures  followed  at  the 
panel  stage.   In  one  district,  attorneys  were  allowed  to 
accompany  parties,  for  example,  but  not  to  participate  in 
the  proceedings. 

One  variation  across  and  even  within  districts 
related  to  differences  in  how  evaluators  were  assigned. 
In  some  places  they  were  assigned  to  specific  schools,  and 
in  other  places  they  evaluated  throughout  the  district. 
As  a  result,  some  teachers  had  only  two  evaluators  during 
the  year,  the  principal  and  an  observer  evaluator,  while 
others  had  three,  four,  or  more.   Since  all  persons 
involved  in  the  evaluation  of  an  individual  had  a  right  to 
be  at  the  hearing  and  might  have  been  needed  to  answer 
panel  questions,  some  teachers  felt  intimidated  by  the 
number  of  people  at  the  hearing  who  were  party  to  the 
decision  under  protest.   Even  though  panels  were 
instructed  to  give  each  side  the  same  time  to  present  its 


case,  the  psychological  impact  of  the  larger  number  of 
persons  on  the  opposite  side  added  extra  stress. 

To  reduce  stress  in  the  future,  several  teachers 
suggested  that  observer  evaluators  and  assistant 
principals  be  located  in  an  adjacent  room  where  they  could 
be  called  for  questioning  about  an  observation  if  they 
were  needed  but  not  be  directly  present  at  the  hearing. 
With  the  increasing  use  of  consensus  ratings  on  the  part 
of  all  evaluators,  however,  such  a  practice  would  deprive 
persons  who  have  a  right  to  be  present,  since  they 
participate  as  equal  partners  in  the  summative  decision, 
to  exercise  that  right.   I  am  inclined  to  favor  the 
inclusion  of  all  persons  who  participate  in  an  evaluation 
as  the  fairest  practice  even  though  some  extra  stress  may 
result  for  the  appellant.   To  alleviate  stress  somewhat, 
however,  appellants  ought  to  be  well  briefed  about  the 
appeals  process  and  offered  assistance  by  a  neutral  party 
in  preparing  for  the  hearing.   In  some  districts  the  CDP 
coordinator  has  attempted  to  assume  that  role. 
Training  of  Panel  Members 

In  each  district  a  pool  of  potential  panel  members  is 
created  from  those  who  have  received  24  hours  of  TPAI 
training  and  special  training  in  the  appeals  process 
itself.   In  some  districts,  this  pool  consists  of  a 
relatively  high  percentage  of  all  teachers  and 
administrators.   In  others,  it  is  a  lower  percent  of 
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teachers  with  administrators  and  observer  evaluators 
making  up  the  rest  of  the  pool. 

Extreme  differences  were  reported  in  the  experience 
and  ability  of  individuals  to  serve  on  appeals  panels. 
Observer  evaluators  and  principals  undergo  not  only  the 
training  but  gain  a  greater  deal  of  practice  in  conducting 
evaluations,  writing  FODIs  and  FODAs.   Many  teachers,  of 
course,  have  no  other  experience  with  the  process  than 
what  the  training  program  and  their  own  evaluations  have 
provided.   Despite  these  background  differences,  however, 
it  is  important  that  appellants  have  a  sufficient  choice 
among  teachers  in  the  pool  to  feel  they  can  truly  select  a 
peer.   The  Appeals  Panel  Training  program  and  materials 
prepared  by  the  Department  of  Public  Instruction  and 
released  in  May,  1988,  should  considerably  enhance  the 
readiness  of  panel  pool  members.   Teachers  need  to  feel 
they  have  been  judged  by  peers  on  appropriate  criteria. 
One  teacher  said  he  felt  better  about  the  whole  career 
development  program  after  going  through  the  appeals 
process,  even  though  he  lost  it;  and  he  intends  to  try 
again  for  a  CSII  promotion  next  year. 

Training  of  panelists  on  the  TPAI  and  the  appeals 
process  is  important  for  overcoming  at  least  two  other 
perceived  concerns  of  appellants.   One  is  a  tendency  of 
some  administrators  to  dominate  panels.   The  other  is  a 
perceived  inconsistency  between  panels  based  on  who  is 
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most  articulate.   Training  can  serve  to  keep  the  focus  on 
what  is  relevant  and  irrelevant  evidence,  what  the 
evidence  shows,  and  whether  prescribed  procedures  are 
followed  rather  than  on  who  is  most  authoritative  or 
persuasive.   It  can  serve  to  remind  panelists  that  they 
have  a  two-fold  responsibility:  (a)  "to  protect  the  rights 
of  the  appellant"  and  (b)  "to  help  ensure  the  quality  of 
performance  of  their  fellow  educators."* 

The  Appeals  Training  program  and  materials  recently 
developed  should  provide  help  in  a  number  of  ways.   They 
specify  very  clearly  the  extent  and  purpose  of  the  review 
and  the  two  issues  that  can  be  appealed,  i.e.  evaluation 
procedures  and  the  possibility  of  an  arbitrary/capricious 
recommendation.   They  provide  a  checklist  of  the  proper 
procedures  to  be  followed  and  cite  examples  of  what  is  and 
is  not  arbitrary  or  capricious.   They  explain  the  nature 
of  evidence  and  provide  clear  examples  of  what  is  relevant 
and  what  is  irrelevant.   They  review  the  nature  of  the 
task  and  procedures  the  panel  should  follow  in  conducting 
its  hearing.   They  provide  examples  of  the  kind  of  report 
the  panel  should  make  to  the  local  school  board,  which 
states  not  only  the  decision  reached  but  the  basis  for 
that  decision. 
Board  of  Education  Actions 

In  contrast  to  the  carefully  prescribed  guidelines, 

*Ibid.,  p.  8. 
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procedures,  and  training  materials  that  have  been 
developed  and  adopted  for  the  conduct  of  panel  hearings, 
almost  nothing  has  been  specified  so  far  regarding  how 
school  boards  are  to  conduct  hearings  and  take  final 
action.   About  all  that  has  been  decided  statewide  is  that 
both  parties  can  have  access  to  counsel  if  they  so  desire. 
Statutes  115C-45  and  115C-305,  as  well  as  longstanding 
traditions,  establish  a  policy  that  all  personnel  actions 
can  ultimately  be  appealed  to  the  local  school  board  for 
decision  and  action.   The  nature  of  school  board  hearings 
and  actions  was  a  subject  of  intense  inquiry  during  my 
visits  to  the  several  pilot  districts  and  my  interviews 
with  board  members,  attorneys,  and  superintendents  in 
particular. 

Even  though  only  38  cases  were  appealed  to  school 
boards  last  year  and  there  were  none  at  all  in  ten  of  the 
sixteen  districts,  considerable  concern  was  expressed 
about  them  by  many  individuals.   Almost  half  of  the 
original  decisions  were  not  upheld. 

Some  board  members  indicated  the  hearings  represented 
their  most  difficult  moments  since  joining  the  board. 
They  were  uneasy  about  how  to  respond  to  personal  calls 
they  received  regarding  individuals  involved  in  the 
hearings.   They  typically  had  not  been  trained  on  the  TPAI 
and  had  no  experience  reviewing  FODIs,  FODAs,  and  teaching 
functions.   They  were  uncertain  about  what  other 
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information  should  be  considered.   I  was  told  by  school 

board  members  themselves,  as  well  as  others,  that 

political  pressures  based  on  who  the  contestants  were  and 

their  standing  in  the  community  and  individual  tendencies 

to  support  the  administration  at  all  costs  or  to  back 

teachers  regardless  of  the  issues  were  often  the  decisive 

influences  in  board  members'  voting.   In  some  instances, 

the  board  felt  manipulated  by  the  lawyers  present,  and  the 

hearings  dragged  on  for  hours  because  information  other 

than  the  current  year's  record  of  performance  and 

evaluation  was   allowed  to  be  discussed.   Teacher 

appellants  wanted  to  use  whatever  information  could  be 

mustered  to  support  their  case,  whether  or  not  it  was 

obtained  through  the  regular  appraisal  procedures. 

Teacher  attorneys  I  talked  with  felt  teachers  deserve  full 

representation  of  counsel  and  a  broad  definition  of 

allowable  evidence.   Hearings  too  often  are  like  a  hostile 

court  room  rather  than  a  setting  for  reasonable  inquiry 

and  review. 

Two  superintendents,  in  particular,  expressed  strong 

concerns  about  potential  damage  to  the  career  development 

program.   One  suggested  using  an  outside  arbitrator  in 

lieu  of  a  board  hearing. 

It's  not  right  to  allow  the  professional  effort  made 
over  a  ten-month  period  to  be  subject  to  overturning 
in  a  one-hour  period  by  a  political  decision.   The 
board  hearing  is  equivalent  to  a  job  loss  hearing  and 
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it  shouldn't  be.   It  has  the  potential  to  destroy  the 
career  ladder.   It  has  polarized  the  faculty  and  upset 
administrator-teacher  relationships . 

To  the  extent  that  school  boards  consider  and  use 

information  in  making  their  decisions  other  than  that 

obtained  through  the  prescribed  performance  appraisal 

system,  I  believe,  they  undermine  the  validity  and 

effectiveness  of  that  system.   Evaluators  would  be  less 

likely  to  rate  teachers  accurately  who  perform  poorly  but 

have  high  status  or  political  influence.   The  evaluation 

process  would  lose  credibility  with  teachers  as  well,  if 

they  see  the  board  overturning  decisions  for  apparent 

political  reasons.   The  potential  damage  to  the  career 

development  program  is  considerable  if  many  cases  are 

overturned  by  the  board.   It  would  basically  mean  that  the 

carefully  designed  and  installed  performance  appraisal 

system  is  not  what  counts  but  whom  one  knows  and  how  much 

influence  one  can  bring  to  bear  on  the  board. 
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CONCLUSIONS  AND  RECOMMENDATIONS 
Is  the  appeals  process  fair  for  all  parties  involved? 

The  major  purpose  of  the  appeals  process  is  to  ensure 
fair  treatment  of  educators  who  feel  they  have  been 
evaluated  improperly  and,  as  a  result,  denied  the  career 
status  and  extra  pay  they  deserve.   They  can  appeal  that 
decision  to  a  panel  of  trained  educators  whom  they  and 
their  evaluator  choose  on  an  equal  basis.   Both  parties 
have  the  same  time  to  present  their  case.   The  teacher  is 
permitted  to  appeal  if  (a)  he  believes  improper  procedures 
were  followed  in  evaluating  him  or  if  (b)  the  summative 
ratings  he  received  were  not  consistent  with  the  evidence 
and,  therefore,  arbitrary  or  capricious.   The  evaluator 
knows  before  the  hearing  is  held  what  is  being  appealed. 
Evidence  to  be  considered  should  bear  directly  on  the 
employee's  current  year  performance  and/or  summative 
appraisal.   Only  evidence  that  is  gathered  prior  to  or  at 
the  summative  evaluation  conference  is  considered 
relevant,  so  panel  members  will  be  examining  the  same  data 
that  presumably  were  used  in  making  the  ratings.   Each 
party  can  have  one  person  present  from  whom  to  seek 
private  advice,  but  that  person  cannot  speak  in  the 
hearing.   Following  the  hearing  panel  members  deliberate 
in  private  in  reaching  a  decision  and  are  instructed  not 
to  discuss  their  proceedings  with  anyone.   The  report  of 
the  panel  should  state  the  reasons  for  the  decision. 
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Copies  go  to  both  parties  as  well  as  to  the  superintendent 
and  school  board  chairman.   A  minority  report  may  be  filed 
by  a  panel  member.   With  all  these  safeguards  and 
guidelines,  the  process  would  seem  to  be  fair  to  all 
parties  through  the  panel  stage.   More  will  be  said  later 
about  the  fairness  of  the  process  when  the  board  hearing 

occurs . 

In  individual  cases,  of  course,  unfair  or  inconsistent 
treatment  may  still  prevail.   A  teacher  may  receive  a 
biased  FODA,  for  example.   He  can  file  a  rebuttal  in 
disagreement,  and  two  earlier  recommendations  of  our  TPAI 
review  (providing  teachers  with  FODIs  before  their  post- 
observation  conferences  and  allocating  a  specific  place  on 
the  FODA  for  evaluatees'  responses)  should  encourage  such 
rebuttals  when  they  are  needed. 

Complaints  were  made  about  scheduling  hearings  after 
school  and  late  in  the  evenings  when  everyone  was  tired. 
Hearings  can  be  quite  stressful  for  all  parties  including 
panelists.   In  one  or  two  instances,  a  teacher  who  was  an 
appellant  one  day  served  as  a  panelist  for  another  teacher 
the  next  day.   I  also  heard  complaints  that  a  person  who 
is  put  on  warning  for  a  year  by  a  poor  sumniative  rating  is 
not  permitted  to  appeal  that  evaluation.   This  ruling 
seems  fair  to  me,  however,  since  no  status  change  or  loss 
of  money  has  yet  taken  place.   He  would  be  able  to  appeal 
the  following  year  if  his  evaluation  does  not  improve  and 
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he  actually  loses  status  and  money.   Program  coordinators 

and  other  administrators  responsible  for  administering  the 

appeals  process  can  reduce  problems  and  stress  by 

requiring  panelists  to  be  chosen  from  other  schools  than 

that  of  the  evaluator  and  evaluatee,  by  being  sensitive  to 

potential  scheduling  conflicts,  and  by  communicating  well 

with  all  parties  about  how  the  hearing  will  be  conducted. 

Obviously,  the  more  people  know  about  what  to  expect,  the 

more  smoothly  hearings  will  run. 

To  what  extent  do  local  boards,  in  reviewing  the 
performance  appraisals,  base  their  decisions  on  information 
obtained  through  the  TPAI  and  the  PPAI? 

As  discussed  earlier,  board  members  and  others  present 

at  the  board  hearings  indicated  that  decisions  in  a  number 

of  cases  were  based,  at  least  in  part,  on  other 

information  than  what  was  obtained  through  the  TPAI  and 

PPAI.   Reports  of  teachers'  performances  in  other  years, 

testimonial  commentary  by  former  students  and  parents, 

students  scores  on  standardized  tests  without  pre-  and 

post-teaching  data  to  permit  valid  comparisons,  and  a  wide 

assortment  of  other  information  was  presented  at  the  board 

reviews,  in  considerable  contrast  to  what  the  peer  panels 

had  examined.   FODIs,  FODAs  and  TPAI  data  were  considered 

as  well,  but  without  board  members  having  the  same 

training  or  experience  in  examining  them  as  the  peer 

panelists.   There  is  no  question  in  my  mind  that  some 

board  members  and  some  local  boards  took  other  information 

«. 
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into  account  in  making  their  final  judgments. 

I  noted  considerable  difference  in  board  member 

attitudes  toward  the  process  in  two  districts.   In  one, 

they  would  have  to  see  blatant,  clear  evidence  in  the  TPAI 

data  that  evaluation  procedures  were  violated  or  judgments 

arbitrary  before  they  would  overrule  a  peer  review 

decision.   In  the  other,  several  board  members  felt  it  was 

particularly  important  to  support  teachers,  given  the 

difficulty  of  their  roles.   They  also  believed  the  TPAI 

does  not  include  all  the  indicators  of  good  teaching. 

Needless  to  say,  considerable  differences  can  be  noted  in 

the  board  decisions  of  these  districts. 

To  what  extent  have  members  of  local  school  boards  had 
sufficient  training  to  review  fairly  and  completely  the 
performance  appraisals? 

According  to  the  January  11  memorandum,  only  four  or 

five  districts  reported  any  orientation  or  training  for 

the  appeals  process,  and  even  here  it  was  undoubtedly 

superficial  and  brief.   Some  board  members  I  talked  with 

indicated  some  familiarity  with  the  TPAI  functions  but 

admitted  difficulty,  when  they  were  preparing  for  the 

hearings,  in  reviewing  FODAs  and  seeing  how  data  there 

related  to  TPAI  ratings.   According  to  one  CDP 

coordinator,  a  board  member  accepted  an  invitation  to  be 

present  for  a  post-conference  following  an  observation 

where  a  FODA  was  discussed,  and  found  it  to  be  one  of  the 
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most  informative  experiences  he  had  ever  had  as  a  board 

member.   This  reaction  obviously  illustrates  the  need  for 

direct  experience  with  the  performance  appraisal  system. 

The  North  Carolina  School  Boards  Association  sponsored 

a  training  session  for  some  of  its  members.   The 

presentation  was  made  by  a  consultant  from  the  Midwest, 

however,  and  was  not  related  directly  to  the  TPAI.   Not 

only  have  most  school  board  members  not  received  training 

on  the  appeals  process,  but  they  undoubtedly  lack  the 

detailed  knowledge  of  the  TPAI  and  the  analysis  procedures 

necessary  to  review  performance  appraisals  fairly  and 

well . 

How  well  does  the  process  work  by  which  a  person 
aggrieved  by  the  teacher  or  principal  performance 
appraisals  may  appeal  and  what  can  be  done  to  improve  that 
process? 

In  summary  of  all  that  has  been  described  or  assessed 

so  far  in  this  report,  let  me  state  first  that  an  appeals 

process  has  been  developed  and  tested  in  several  pilot 

districts  which  seems  to  be  working.   Although  the  panel 

hearings  may  be  stressful  for  the  parties  involved  and 

even  the  panelists,  teachers  at  least  have  an  opportunity 

to  get  a  second  judgment  on  whether  they  were  evaluated 

properly  and  judged  fairly.   In  approximately  one-fifth  of 

the  cases,  their  appeal  was  successful  and  the  denial  of 

career  status  rescinded.   Even  some  of  those  who  were 

unsuccessful  felt  better  about  the  performance  appraisal 

system  and  career  ladder  because  they  went  through  the 
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appeals  process. 

The  appeals  process  is  an  essential  component  of  the 
career  development  program.   It  protects  the  rights  of 
individuals  against  biased  and  arbitrary  judgment  and 
unfair  treatment  in  the  assessment  of  their  performance. 
Such  protection  should  strengthen  the  validity  of  the  PAS 
as  well  and  enhance  its  effectiveness  in  improving 
instruction. 

The  appeals  process  has  already  improved  the  quality 
of  summative  ratings  and  feedback  to  teachers.   Several 
principals  commented  that  as  a  result  of  having  their 
statements  challenged  during  appeals  hearings  they  have 
been  more  careful  this  year  in  the  phrases  they  use  on 
FODAs  and  in  post-observation  conferences  to  convey 
strengths  and  weaknesses  accurately  so  teachers  will  know 
how  they  are  perceived  as  the  year  goes  along  and  not  be 
surprised  by  the  summative  rating.   In  one  district,  a  new 
procedure  was  started  this  year  of  sending  teachars  a 
midyear  report  after  two  observations  to  inform  them  in 
general  terms  how  they  were  doing  so  they  would  receive 
ample  warning  of  areas  that  needed  strengthening.   In  most 
pilot  districts,  evaluators  attempted  to  generate  a  list 
of  key  terms  that  could  be  used  consistently  to  refer  in 
precise  ways  to  the  assessed  quality  of  teaching  actions. 
Such  efforts  should  not  only  improve  the  interreliability 
of  ratings  but  also  increase  the  accuracy  of  feedback  to 
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teachers  on  how  their  performance  is  perceived. 

Not  all  of  the  effects  of  the  appeals  process  so  far, 
however,  have  been  positive.   For  some  principals,  the 
board  hearing  represented  the  first  time  their  judgment 
had  been  called  into  question  in  a  public  or  semipublic 
forum.   Questioning  by  an  attorney  did  not  lighten  their 
task.   To  the  extent  that  inconsistencies  showed  up  in 
their  written  and  oral  statements,  they  tended  to  look 
foolish.   School  board  members,  too,  reported  being  uneasy 
and  feeling  they  were  in  an  adversial  situation  with  the 
administration . 

If  school  boards  reverse  very  many  personnel  decisions 
of  this  sort,  especially  in  ways  contrary  to  what  the  PAS 
indicates,  the  entire  effort  to  conduct  sound  performance 
assessment  will  be  threatened.   Principals  will  be  less 
willing  to  evaluate  rigorously.   Teachers  will  be 
reinforced  for  complaining  loudly  about  anything  less  than 
high  ratings.   And  whatever  fairness  and  objectivity  the 
performance  appraisal  system  brings  to  the  evaluation 
process  will  be  lost. 

Inconsistencies  still  exist  in  the  rigor,  objectivity, 
and  validity  of  performance  assessment — from  one  school  to 
another  and  one  school  district  to  another.   Some 
districts  average  almost  one  scale-step  higher  than  other 
districts.   Yet,  by  other  measures,  such  as  student 
achievement  scores  and  dropout  or  absentee  rates,  there  is 
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no  indication  of  better  overall  teaching  in  those 
districts  with  the  highest  ratings.   To  the  contrary,  I 
propose,  those  districts  doing  the  best  job  of  evaluating 
teaching  may  have  the  lowest  average  teacher  performance 
ratings  and  the  highest  success  rate  on  student 
performance  indicators.   I  believe  I  visited  such  a 
district.   The  percentage  of  teachers  eligible,  in  terms 
of  years  of  teaching  experience  in  the  district,  who  were 
actually  promoted  to  CSII  level  was  only  57  compared  to  as 
high  as  87  in  other  pilot  districts.   It  seems  to  me  that 
the  closer  a  district  is  to  promoting  all  eligible 
teachers  to  the  higher  rank  the  greater  one's  claim  to  a 
right  to  be  promoted.   A  high  percentage  (e.g.  87%)  may 
provide  a  strong  case  that  one  has  been  discriminated 
against  for  personal  bias  reasons  and  weaken  the  claim 
that  promotion  is  based  on  a  sound  measure  of  teaching 
performance. 

Differences  are  apparent  also  between  schools  within 
districts.   In  some  schools,  no  teachers  eligible  and 
applying  for  promotion  in  1986-87  received  ratings  low 
enough  to  deny  it.   In  other  schools,  up  to  a  third  of 
those  eligible  did  not  achieve  the  CSII  level.   Principals 
from  the  latter  schools  had  to  defend  their  summative 
ratings  in  appeals  hearings  at  considerable  time  and 
effort.   Principals  with  no  teachers  appearing  because 
none  were  denied  promotion  had  a  much  easier  time.   Some 
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principals,  on  principle,  denied  they  had  any  inferior 
experienced  teachers  and  assigned  no  ratings  less  than 
five  on  a  six-point  scale.   In  our  earlier  review  of  the 
TPAI  and  PPAI  system,  I  and  my  colleagues  recommended  that 
the  teacher  evaluation  practices  of  principals  be 
monitored  by  having  them  justify  higher  or  lower  average 
ratings  than  what  other  principals  give.   Their  own 
evaluation  as  instructional  leaders  ought  to  depend  in 
part  on  how  well  they  conduct  the  teacher  performance 
assessment.   Those  who  do  a  good  job  of  assessing  teacher 
performance  ought  to  be  rewarded  for  it.   Those  who  do  a 
poor  job  or  opt  out  by  assigning  nothing  but  5  and  6 
ratings  ought  to  be  questioned  about  it  and,  if  they 
cannot  justify  their  ratings,  be  downgraded  themselves. 
Performance  assessment  is  a  demanding  responsibility  and 
principals  should  be  judged  in  part  on  how  well  they  do 
it.   It  is  most  important  that  those  in  the  career 
development  program  be  evaluated  fairly  but  rigorously. 

Several  recent  developments  should  improve  the  quality 
of  performance  evaluations  and  reduce  the  number  of 
complaints.   One  is  the  stabilizing  influence  of  the 
observer  evaluators.   Increasingly,  OEs  are  involved  as 
direct  contributors  to  the  summative  ratings.   To  the 
extent  they  interact  with  principals  and  assistant 
principals  in  making  these  ratings,  professional  judgment 
overall  is  enhanced.   Because  of  their  recent  experience 
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as  classroom  teachers  and  their  full-time  assignment  as 
evaluators,  they  soon  become  the  greatest  source  of 
expertise  on  teacher  evaluation  in  a  school  district. 
Their  visits  to  many  schools  and  classrooms  give  them  the 
background  experience  and  independence  of  judgment  to 
question  administrators  about  seemingly  unjustified 
ratings  and  to  help  achieve  consensus  ratings  of  teacher 
performance.   During  this  past  year,  making  summative 
ratings  by  consensus  became  standard  practice  in  the  pilot 
districts.   As  principals  came  to  accept  OEs  as  full 
partners  in  the  process,  the  quality  of  summative  ratings 
improved  immensely.   The  need  for  appeals  should  decline 
as  well. 

Another  major  development  is  the  production  of  good 
training  materials  by  the  Department  of  Public 
Instruction.   Part  of  the  reason  for  so  little  training  of 
school  board  members  in  particular  was  the  lack  of 
training  materials.   The  videotape  and  supporting 
materials  published  in  May,  1988,  now  provide  clear 
directives  about  the  roles  of  those  involved  in  appeals 
hearings,  the  grounds  on  which  appeals  can  be  made,  and  a 
host  of  other  specific  guidelines  for  the  conduct  of 
hearings. 

It  is  imperative  that  persons  involved  in  the  appeals 
process  be  well  trained  for  that  task.  Materials  are  now 
available  for  that  to  happen.   One  of  the  biggest  threats 
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to  the  career  development  program  at  this  time  and  the 
performance  appraisal  system  that  undergirds  it  is  the 
lack  of  (a)  specific  knowledge  by  school  board  members  of 
the  performance  appraisal  system  and  (b)  direction  and 
procedures  for  school  board  hearings  of  those  who  were 
denied  career  status. 

My  major  recommendation  is  that  school  boards  develop, 
in  conjunction  with  the  State  Board  of  Education, 
agreements  about  procedures  to  be  followed,  criteria  to  be 
used,  evidence  to  be  considered  and  evidence  that  is 
irrelevant  and  not  to  be  considered  in  the  appeals 
process.   I  hope  that  school  boards  will  agree  to 
guidelines  similar  to  those  used  by  peer  panels,  so  their 
review  will  bring  lay  persons'  judgments  to  bear  on 
essentially  that  same  data  that  professional  educators 
have  considered.   To  do  otherwise  only  increases  ambiguity 
and  inconsistency  in  the  whole  process;  and  potentially  it 
could  destroy  the  validity  of  the  system. 

All  who  are  involved  in  the  review  process  ought  to  be 
trained  not  only  in  the  appeals  procedure  but  in  the 
performance  appraisal  instruments  as  well.   With  the 
recent  availability  of  good  training  materials,  the  School 
Boards  Association  ought  to  establish  regular  and  ongoing 
training  clinics  for  those  who  are  involved. 
Consideration  might  also  be  given  by  local  boards  to 
delegate  the  hearing  responsibility  to  two  or  more  of  its 
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members  who  agree  to  participate  in  the  training,  become 
informed  about  the  process,  and  conduct  the  actual 
hearings  on  behalf  of  the  board. 


27 


FINAL  LEGAL  CONCERNS 
Two  underlying  issues  of  a  legal  nature  complicate  my 
recommendations.   Both  suggest  caution  in  what  I  propose 
and  a  need  for  authoritative  legal  opinion  at  the  very 
least. 

First,  a  core  issue  underlying  the  entire  program  is 
whether  evaluatees  have  a  full  property  right  in  a 
promotion  or  only  a  property  interest.   A  promotion  with 
pay  raise  is  at  stake,  not  one's  job.   Those  who  designed 
the  appeals  process  and  most  attorneys,  board  members,  and 
superintendents  I  talked  with  believe  that  the  appeals 
process  does  not  involve  a  property  right  and  that  full 
due  process  as  a  court  of  law  might  provide  is  not 
appropriate.   The  denial  of  a  promotion  is  not  the  same  as 
the  loss  of  a  job  by  a  tenured  teacher.   A  statement  in 
statute  115C-363  indicates  that  "an  involuntary 
reclassification  may  not  be  considered  a  demotion"  from 
the  tenured  teacher  ranks.   Therefore,  it  would  seem  that 
the  greater  the  similarity  of  school  board  and  peer  panel 
hearings  in  procedure  and  criteria,  the  greater  the 
consistency  and  integrity  of  the  appeals  process. 

Second,  various  means  were  suggested  to  alleviate 
stress  for  panelists  caused  by  overlapping  personal 
relationships  in  close  communities  and  to  prevent  board 
politics  from  interfering  with  the  appeals  process. 
Outside  arbitrators  could  provide  the  final  hearing,  if  a 
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board  so  chose,  or  panels  could  come  from  other  districts 
or  a  pool  of  trained  board  members  from  across  the  state. 
Communities  could  be  given  various  alternatives  regarding 
the  composition  of  hearing  panels  if  such  variation  does 
not  run  counter  to  the  consistency-of-treatment  principle 
that  a  statewide  personnel  system  requires.   What  works 
well  in  some  communities  may  not  work  so  well  in  others. 
I  would  recommend  providing  alternatives  if  legal 
authorities  feel  the  equal  treatment  principle  would  not 
be  compromised. 
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